Information capacity of stochastic pooling networks is achieved by discrete inputs
نویسندگان
چکیده
منابع مشابه
Stochastic Pooling Networks
We introduce and define the concept of a stochastic pooling network (SPN), as a model for sensor systems where redundancy and two forms of ‘noise’ – lossy compression and randomness – interact in surprising ways. Our approach to analyzing SPNs is information theoretic. We define an SPN as a network with multiple nodes that each produce noisy and compressed measurements of the same information. ...
متن کاملDiscrete Time Stochastic Networks
In this section we study the relation between throughput and queue-length distributions in discrete-time closed cyclic networks when the number of customers and the number of nodes grows unboundedly. To state our findings more precisely: For each positive integer N , consider a cyclic network with n(N) nodes and k(N) customers. We suppose that there are only finitely many types of nodes. That i...
متن کاملStochastic Pooling for Regularization of Deep Convolutional Neural Networks
We introduce a simple and effective method for regularizing large convolutional neural networks. We replace the conventional deterministic pooling operations with a stochastic procedure, randomly picking the activation within each pooling region according to a multinomial distribution, given by the activities within the pooling region. The approach is hyper-parameter free and can be combined wi...
متن کاملShannon Information Capacity of Discrete Synapses
X iv :0 80 3. 19 23 v1 [ qbi o. N C ] 1 3 M ar 2 00 8 Shannon Information Capa ity of Dis rete Synapses Adam B. Barrett and M.C.W. van Rossum Institute for Adaptive and Neural Computation University of Edinburgh, 5 Forrest Hill Edinburgh EH1 2QL, UK There is eviden e that biologi al synapses have only a xed number of dis rete weight states. Memory storage with su h synapses behaves quite di ere...
متن کاملInformation storage capacity of discrete spin systems
What is the limit of information storage capacity of discrete spin systems? To answer this question, we study classical error-correcting codes which can be physically realized as the energy ground space of gapped local Hamiltonians. For discrete spin systems on a D-dimensional lattice governed by local frustration-free Hamiltonians, the following bound is known to hold; kd 1/D ≤ O(n) where k is...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Physical Review E
سال: 2009
ISSN: 1539-3755,1550-2376
DOI: 10.1103/physreve.79.041107